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c-j . Abstract 

Various versions of the Dynamical Systems Method (DSM) are proposed for 
solving linear ill-posed problems with bounded and unbounded operators. Conver- 
gence of the proposed methods is proved. Some new results concerning discrepancy 
principle for choosing regularization parameter are obtained. 

> 

1 Introduction 

In this paper part of the results from the author's invited talk at ICMAAS06 are pre- 

O 
^— > 



sented. This part deals with linear ill-posed problems. 

Some of the results in this paper are taken from the papers of the author cited in the 
bibliography and many results are new. The DSM method is developed in |8j-|28|. 
The discrepancy principle, discussed earlier in jHj, has been studied recently in [8]-[T0] 
and in [Zj. 

Consider an equation 

Au-f = 0, (1.1) 

where A is an operator in a Banach space X. If A is a homeomorphism of X onto X 
(i.e., continuous injective and surjective map in X which has a continuous inverse map) 
then problem is called well-posed in the Hadamard's sense. Otherwise it is called 
ill-posed. 

The Dynamical Systems Method (DSM) for solving equation (jl.lj) consists of solving 
the Cauchy problem 

du 

u = $(*,«), u(0) = u ; "=-^I> ( L2 ) 
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which we call a dynamical system, where $ is chosen so that 



3\u(t) Vt > 0; 



3w(oo); 



A(u(oc)) = f. 



(1.3) 



Here tt(oo) := lim^ 00 M(i). We do not assume that the solution to is unique, but 
we do assume that there exists a solution to (JTTTJ. If (jl.3j) holds, then we say that DSM 
is justified for solving equation (jl.lj) . 

There is a large body of literature on solving ill-posed problems (see e.g., j2], [Zj, 
29] and references therein). Variational regularization, iterative regularization, quasiso- 
lutions and quasiinversion are some of the methods for stable solution of ill-posed prob- 
lems discussed in the literature. In this paper several new methods for stable solutions of 
linear ill-posed problems is discussed. They are based on the dynamical systems method 
(DSM). We describe the DSM for solving ill-posed problems. This method was developed 
relatively recently as a method for solving a wide variety of linear and nonlinear ill-posed 
problems [E]-[2E| although it was proposed already in |1. for solving well-posed problems. 

Because of the space limitations we will not discuss solving nonlinear ill-posed prob- 
lems by the DSM in this paper, and refer the reader to jT], [T5] . 

Here we describe a new version of DSM for solving linear ill-posed problems. There 
are many practical problems of this type. We only mentioned solving ill-conditioned 
linear algebraic systems and Fredholm equations of the first kind. The novel points in 
our results are not only the method of solving these problems by DSM but also the 
applicability of the method to unbounded operators. In the literature a widely-discussed 
method for solving ill-posed problem (jl.lj) is the method of variational regularization 
introduced by Phillips (p\) and studied by Tikhonov (|2H])5 Morozov (0), Ivanov (|2|), 
and many other authors under the assumption that the operator A in (jl.lj) is a linear 
bounded operator. There are some results on regularization of unbounded operators also 
(e.g., jl], [2E])- The variational regularization method for stable solution of (jl.lj) consists 
of solving the problem 



where a > is a constant, called a regularization parameter, fs are the "noisy data", i.e., 
an element which satisfies the inequality ||/<$ — /|| < S and which is given together with 
the "noise level" 5 > 0, while the exact data / are not known. The stable solution to 
(jl.lj) is an element us such that lim^o H M <5 — 2/11 = 0, where Ay = f and y is the unique 
minimal-norm solution of the linear equation (jl.lj) . If X is a Hilbert space H, which 
we assume below, then the minimal-norm solution is the solution which is orthogonal to 
the null space N of A, N = N(A) = {u : Au = 0}. If the linear operator A in (jl.lj) is 
unbounded, then we assume that it is closed and densely defined, so that its adjoint A* 
is densely defined and closed (see, e.g., 

If A is bounded, then the necessary and sufficient condition for the minimizer of the 
quadratic functional (jl.4j) is the Euler's equation 



F(u) := \\Au — fs\\ 2 + a|M| 2 = min, 



(1.4) 



T a u = A*f s , T a : 



T + al, T:=A*A, 



(1.5) 
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where I is the identity operator and T > is a selfadjoint operator. Equation (jl.5|) has 
a unique solution u a> s = T~ 1 A*f$. One can choose a = a(5) so that lim^ a(5) = and 
us := M a (<5),<5 is a stable solution to 

lim || iz$ — 2/ 1 1 = 0; (1-6) 

where Ay = 0, y _L N. There are a priori choices for choosing a(S) and a posteriori ones. 
An a priori choice is based on the estimate: 

K^fs - V\\ < IK'A^fs -f)\\ + \K*A*f -y\\<JL + r](a), (1.7) 

where 

V 2 (a) = \\T^Ty - yf = a 2 f° - ||P^f as a - 0, (1.8) 

and is the orthoprojector onto N. Since we assume that y _L N, equation ()1.8|) implies 

limr?(a) = 0. (1.9) 
The term in (jl.7j) appears due to the estimate: 



1.10) 



s >o s + a 

Here Q := AA*, U is a partial isometry, Q a '■= Q + a-I, we have used the formula 

T-'A* = A*Q~\ (1.11) 

the polar decomposition: A* = UQ^, and the spectral theorem for the selfadjoint oper- 
ator Q, namely ||#(Q)|| = sup s>0 |y(s)|. Formula (jl.llj) is obvious if A is bounded: mul- 
tiply (jl.llj) by T a from the left and then by Q a from the right, and get A*(AA* + al) = 
(A*A + aI)A*, which is an obvious identity. Since the operators Q a and T a are boundedly 
invertible, one may reverse the steps and get (jl.llj) . Thus a priori choices of a(5), which 
imply (jl.6j) . are: 

lim - = 0, lim a(5) = 0. (1.12) 

There are many functions a(5) satisfying (jl,12j) . One can find an optimal value a(S) 
by minimizing the right-hand side of (jl.7j) with respect to a. Alternatively, one may 
calculate a(5) by solving the equation 5 = 2yJ a(8)r)(a) for a for a fixed small 5 > 0. 

If A is closed, densely defined in H, unbounded, and a = const > 0, then the author 
has proved that the operator T~ l A*, with the domain D(A*), is closable, its closure, 
denoted again T" 1 ^*, is a bounded operator defined on all of H, \\T~ 1 A*\\ < and 
(II. lj) holds. For convenience of the reader let us sketch the proof of these claims. To 
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check that T a 1 A* is closable, one takes h n G D(T a 1 A*) = D(A*), such that h n — > and 
T~ 1 A*h n — > g as n — > oo, and checks that g = 0. Indeed, let w G if be arbitrary. Then 

(g,u) = lim (T- 1 ^^^) = lim (h n , AT~ l u) = 0. (1.13) 

n— >oo n— >oo 

Since w is arbitrary, this implies g = 0, as claimed. Note that T" 1 ^ G -D(A), so that the 
above calculation is justified. If one drops the index n and the lim in (j!.13j) . then one 
can see that the adjoint to the closure of T~ X A* is the operator AT' 1 , defined on all of 
H and bounded 

||Ar-1 = iic/rir-ii < ^. 

Since ||A*|| = \\A\\, one gets HT'M*!! < Finally formula llj) can be proved 

for unbounded closed, densely define operator A as above, if one checks that the oper- 
ator A*AA* is densely defined. This operator is densely defined because the operator 
A* A A* A = T 2 is densely defined if T is, and D(T 2 ) C D(A*AA*). 

Let us now describe an a posteriori choice of a(5) which implies (jl.fi j) and which is 
called a discrepancy principle. This principle was discussed in [SJ , [Zj - It consists in 
finding a (5) from the equation 

\\Au a , s -f s \\=Cd, KC<2, (1.14) 

where C = const, \\Fs\\ > C5 and u a j = T~ 1 A*f$. One can prove (e.e. see [Zj) that the 
equation (jl,14jl for a small fixed 5 > has a unique solution a(S), lim^o 0,(5) = 0, and 
us = M a (5),<5 satisfies (jl.6|) . i.e. 115 is a stable solution to (jl.lj) . To prove these claims one 
writes (jl,14j) as 

= UfAT- 1 ^ - /]M| 2 = Hfgg- 1 - /]M| 2 = H ^iMiM := h{atS)i (L i 5 ) 

j + aj 

and takes into account that /i(a, 5) is, for a fixed 5 > 0, a continuous monotone function 
of a, h(oo, 5) = \\fs || 2 > C 2 <5 2 and /i(+0, 5) = ||Pjv*/,$|| < 5 2 , so that there exists a unique 
a = a(5) such that h(a(5),5) = C 2 5 2 . Here we have denoted N* := N(A*), used the 
obvious relation N(Q) = N(A*), the inequality \\P N *fs\\ < \\PN*(fs ~ f)\\ + ll-FWII < 
Wfs ~ /|| — &i an d the relation P^*f = 0. This last relation follows from the formulas 
/ G R(A) and R(A) 1 N*. 

Let us now check that if a(5) solves (jl.5j) then us = ct a (S),s satisfies (jl.Sjl . One has 
F(us) < F(y), so 

\\Aus - fsf + a(5)||^f < 5 2 + a(<*)||3/|| 2 . (L16) 
Since ||Aw<5 — /5II 2 = C 2 5 2 > 5 2 and a (5) > 0, one gets 

Nil < \\v\l (i.i7) 

Therefore one can select a weakly convergent sequence w n = us n , u n — it, as n — > 00. 
Let us prove that u = y and lim^oo ||w n — y|| = 0. Since this holds for any subsequence, 
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it follows that (jl.6j) holds. To prove that u = y note that (|1.17j) implies ||it|| < ||y||, and 
that u solves (jl.lj) . Since the minimal-norm solution to is unique, it follows that 
u = y. To check that u solves (jl.lj) we note that lim^o \\Au$ — /|| = as follows from 
(I1.16J1 because lim^ cl(5) = 0. The relations us — ^ u and \\Aus — f\\ — > as 5 — ► imply 
Au = f and lim^o \\ u s — u\\ = 0. Let us first check that Au = f. One has 

(f,g) = lim(Aus,g) = (u,A*g) Vg G D(A*). 

o— >0 

Thus u G D(A) and Au = f, as claimed. As we proved above, this implies that u = y. 
Therefore u$ — ^ y, \\us\\ < \\y\\. This implies that lim^o \\ u s — y\\ = 0. Indeed, 

\\u s -y\\ 2 = \\u s \\ 2 + ||n|| 2 - 2Re(u s ,y) < 2\\y\\ 2 - 2Re{u s ,y) ^0 as 5 -> 0. (1.18) 

Thus the relation (jl.6|) is proved for the choice of a(S) by the discrepancy principle. 

The drawback of the a priori choice if a(S) is that it is nonunique and although it 
guarantees convergence ([1.16)1 . the error of the method can be large if 5 > is fixed. The 
drawback of the discrepancy principle is the necessity of solving the nonlinear equation 
(11.14)1 and also a possible large error for a fixed S. 

In Section 2 we discuss the DSM for solving linear equations (jl.lj) . 



2 DSM for solving linear problems 

We assume first that the linear closed densely defined in H operator in (jl.lj) is selfadjoint, 
A = A*. This is not a restriction: every solvable linear equation (jl . lj) is equivalent to 
an equation Tu = A*f, where T = T* = A* A. Indeed, if Au = f, then applying 
A* and assuming / e D(A*), one gets Tu = A*f. Conversely, if Tu = A*f, and 
/ = Ay, then Tu = Ty. Multiply the equation = T(u — y) by u — y and get = 
(A*A(u-y),u-y) = \\Au-Ay\\ 2 . Thus Au = Ay = f. If A is bounded, then / G D(A*) 
for any / G H. If A is unbounded, then D(A*) is a linear dense subset of H. In this 
case, if / D(A*), then we define the solution of equation Tu = A*f by the formula 
u = lim a ^o T~ 1 A*f. As we have proved in Section Q for any / G R(A) this limit exists 
and equals to the minimal- norm solution y: lim a ^o T~ l A*Au = y if Au = f. This is true 
because lim a ^ T~ x Tu = u — P^u = y. 

The DSM for solving equation (jl.lj) with a linear selfadjoint operator can be con- 
structed as follows. Consider the problem 

du 

u a = i(A + ia)u a -if, u(0) = 0; u = —, (2.1) 

dt 

where a = const > 0. Our first result is formulated as Theorem ^ 
Theorem 1. If Ay = f and y _L iV ; then 

lim lim uJt) = y. (2.2) 

a^0 t—*oo 
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Our second result shows that the method, based on Theorem 1, gives a stable solution 
of the equation Au = f. Assume that \\fg — f\\ < 8, and let u aj g(t) be the solution to 
(j2.1|) with fs in place of /. 

Theorem 2. There exist t = tg, limg^otg = oo, and a = a(5), lim^o 0,(8) = 0, such that 
ug ■= u a (s),s(ts) satisfies (JO). 

We will discuss the ways to choose a (8) and tg after the proofs of these theorems are 
given. 

From the numerical point of view if one integrates problem (j2.1j) with the exact data 
/ on the interval < t < T, and T is fixed, then one is interested in choosing such 
a = a(T) that lim^oo \\u a (r)(T) — y\\ = 0. We will give such a choice of a(T). 

Before we start proving these two theorems, let us explain the ideas of the proof. 
Suppose B is a linear operator and its inverse B" 1 exists and is bounded. 

Finally, assume that 

lim ||e B *|| = 0. (2.3) 

t— >oo 

This will happen, for example, if ReB < — c, c > is a constant. Under these assump- 
tions one has 

e Bs ds = B' 1 (e Bt -I), -Mm e Bs ds = B~ l . (2.4) 

The operator J Q * e Bs ds solves the problem 

W = BW + J, W(0)=0, (2.5) 
where I is the identity operator. If lim^oo ||e Bi || = 0, then 

- lim W(t) = B-\ (2.6) 

t^oo 

The basic idea of the DSM is the representation of the inverse operator as the limit 
as t — >• oo of the solution to the Cauchy problem (|2.5|) . 
Let us prove Theorem ^ 

Proof of Theorem^ The solution to (J2.1)) is 

Ua (t) = f e l{A+ia){t - s) (if)ds = [i(A + to)}" 1 ( e ^ A+ia)t - i) (-if). 
Jo 

Since || e i ( j4+M )* || = e~ at — > as t — > oo, one gets 

\imu a (t) = (A + iaY l f. (2.7) 

Since / = Ay, one has 

r](a) := \\(A + ia)~ 1 Ay - y\\ = a\\(A + ia)~ 1 y\\ — > as a -> 0. (2.8) 



6 



The last relation follows from the assumption y _L N. Indeed, by the spectral theorem 
one has: 



lim rj 2 (a) = lim a 2 \\ (A + ia) 1 y u2 

a— >0 a^O 



x a 2 



lim / -——d(E s y,y) = \\(E o -E o „ o )y\\ 2 =\\P N y\\ 2 = 0. 
a ^°J-oo s 2 + a 2 



Theorem is proved. □ 

Remark 1. In a numerical implementation of Theorem^ one chooses r, and a = a{r), 
and integrates (|2-1J) on the interval [0,r]. One chooses r so that 

lim ||u ( T ) - y\\ = 0. (2.9) 

T— >CJO 

In order to choose a(r), note that \\u a (t) —u a (oo)\\ < as follows from the derivation 
of (|2.7j) . Therefore, by Theorem^ the relation ()2.9|) holds if 

e -a(r)r 

lim = lim a(r) = 0. (2.10) 

t — >oo 0(t) t^oo 

For example, one may take a(r) = r~ 7 ; where < 7 < 1 is a constant. 
Proof of Theorem 2. Let us start with the formula: 

««,«(*) = f e i{A+ia) «- s) (-if s )ds = [i{A + ia)\- 1 ( e < A+ia *-l) (if 6 ). (2.11) 

Thus 



One has 



S := \\u a>5 (t) - 2/|| < ||M Q>(5 (t) - u B (t)|| + \\u a (t) - y\\. (2.12) 

\\u a At)-u a (t)\\<^^\\I-e iAt - at \\<-, (2.13) 

a a 



e -at 



and 

IM*)-3/||<— 11/11 + 77(a), (2-14) 
a 

where 77(a) is defined in (2.8), lim a ^ 77(a) = 0. Since ||/|| < + 5 < c, one obtains 
from ([fp2 >(j2~TI)l : 

lim£ = (2.15) 

<5^0 

provided that t = ts, a = a(S) and 

e -<*(5)t« J 

limt^ = 00, lima(<5) = 0, lim — — — = 0, lim— — = 0. (2.16) 

5^0 5^0 a(8) s^o a(5) 

Theorem |21 is proved. □ 
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Remark 2. There are many choices of tg and a(S) satisfying relations (|2.16|) . If one 

has an estimate of the rate of decay 0/77(0) as a — > 0, then one may obtain some rate 
of convergence of S to zero as 5 — > 0. However, it is impossible, in general, to get a 
rate of decay of r/(a) as a — > without additional assumptions on the data f or on the 
solution y. A typical assumption is y = Az, that is, y G R(A). If fractional powers of A 
are defined (which is the case when A > 0, for example) then one may assume y = A J z, 
7 > 0. Let us show how to get the rate of decay ofi](a) under such assumptions. Assume, 
for example, that y = Az. Then 



00 aV 



V (.<>■)= I ?AE a z,z) <a 2 \\z\\ 2 } (2.17) 

and the error bound is 



-00 a + s * 



25 e~ at 

S<— + Cl + c 2 a, c 2 =|MI, c 1 = \\f s \\ + 5. (2.18) 

a a 

Choose, for example, a = 5 1 , < 7 < 1, and t$ = <5~ M , ^ > 7. Then lim^ e = 
and (12.15)) holds with the rate 5 U , v = mm(l- 7, 7). If r y = | then max 0<7<1 min(l — 7, 7) 
is equal to | ; and for 7=5 one gets E = 0(5 1 ^ 2 ) if t$ = 5~^, /x > |. 

3 Second version of the DSM 

Consider problem ()2.1|) with a = a(t). Let us assume that a(t) > is a continuous 
function monotonically decaying to zero as t — > 00 and 

poo 

0<a(t)\0, a' + a 2 eL l {0, 00), / a(s)ds = 00. (3.1) 

Jo 

The solution to this problem is 

u(t)= [ e lA{t - s) SX p)dp ds{-if). (3.2) 
Jo 

Theorem 3. Under the above assumptions one has 

lim \\u(t) - y\\ = 0. (3.3) 

Proof of Theorem 3. Since / = Ay, one gets, integrating by parts, 

ft 

u {t) = e iAt e- iAs -^ adp y\l - \ e lA{t ~ s) a(s)e- ^ a{p)dp dsy. 



JO 

Thus 

ft 

u {t)=y-e iAt ~^ adp y~ e iA{t - s) a(s)e-f> dp dsy, (3.4) 

Jo 
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and 

i-t 

\\u(t) - y\\ < e-fi^WyW + || / e U( - t ^a{s)e- ^ adp dsy\\ := J x + J 2 . 

Jo 

By the last assumption (|3.1|) one gets 

lim J 1 = 

t^oo 



We now prove that 



Using the spectral theorem, one gets 



lim J 2 = 0. 



J 2 



d(E x y, y) 



e iX{t ~ s) a(s)e~^ adp ds 



(3.5) 

(3.6) 
(3.7) 

(3.8) 



Let us prove that 



lim 

t— >oo 



e iX{t - s) a(s)e~^ adp ds = 0, VA ^ 0. 



(3.9) 



From (|3.8j) . (|3.9|) and the assumption y _L N, the conclusion ()3.7|) follows. 
Let us verify (|3.9|) . Integrating by parts one gets: 



Thus 



e iX{t - s) a(s)e 



^ adp ds ' 



i\(t-s) 



a(s)e-fs adp 



+ 



iX 



'^-^ [a'(s)+a 2 (s)}e- £ a ^ dp ds. 

(3.10) 



a(t) e iXt So adp a(0) , , 

^3 = ^7T + - — + Ji, 0, 



(3.11) 

— IA IA 

where J4 denotes the last integral in (|3.10|) . The first two terms in (|3.11j) tend to zero as 
t — > 00 because of the assumptions about a(t). 
Assumptions (|3.1|) imply that 



lim Ja = 0. 



t— »oo 



Thus, Theorem |3] is proved. 



(3.12) 
□ 

— — , where Cq,Ci > are arbitrary 



Remark 3. For example, the function a{t) = ^ Ci+t ^ b 
constants and b G (|, 1) is a constant, satisfies assumptions (13.1)1 . 

It is interesting to check numerically the efficiency of the algorithm based on Theo- 
rem^ 



Let us prove that Theorem El yields a stable solution to equation (jl.lj) . 
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Theorem 4. There exists a stopping time t$, lim^o^ — °°> such that holds with 
Us = u$(ts), where Us{t) is the solution to problem (J2.1j) with a = a(t) and fg in place of 
f,\\fs-f\\<8. 

Proof of Theorem^ One has 

\\u s (t) - y\\ < \\u s (t) - u(t)\\ + \\u(t) - y\\, (3.13) 

where u(t) solves problem (|2.1|) with a = a(t) and exact data. We have proved in 
Theorem |3] that 

lim \\u(t) -y\\ = 0. (3.14) 

We have 

IM*)-u(t)|| < f e-^ a{p)dp ds\\f s -f\\ < * (3.15) 
Jo a l r J 

Here the estimate Jq e~ ^ adp ds < was used. This estimate is derived easily: 

f e-f° adp ds < / a(s)e~^ adp dp = J-e-J"*"* = ( 1 - e - ^ adp \ < 
Jo a \t) Jo a>\t) a{t) V / a(t) 

Choose ts so that 

\imt$ = oo, lim — — - = 0. (3.16) 

.5-0 5-^0 a(t$) 

This is obviously possible to do. 
Then (j3~T3 ]) -fTTH |l imply 

lim \\u s {t s ) - y\\ = 0. (3.17) 

8^0 

Theorem 0] is proved. □ 



4 Third version of DSM for solving equation (1.1) 

u = -u + T^A*f, u(0) = 0, (4.1) 

where / = Ay, y J_ N, and a(t) > is a monotonically decaying continuous function such 
that lim^oo a(t) = and J °° a(t)dt = oo. We could take the initial condition it(0) = u 
arbitrary. The contribution to the solution to problem (|4.1j) which comes from the initial 
condition u is the term u e~*. It decays exponentially fast and our arguments do not 
depend on this term essentially. To simplify and shorten our argument we take Uq = 0. 

Theorem 5. lim u(t) = y. 
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Proof of Theorem 5. One has 
u(t) 

Thus 



f e-^T-^Ayds= f e^yds - f e~^ a{s)T^ yds. 
Jo Jo Jo 



\\u(t) - y\\ < e-'IMI + f e-^a(s)\\T-l )y \\ds. (4.2) 

J 

One can easily check that if b(s) is a continuous function on [0, oo) and b(oo) = lim^oo b(s) 
exists, then 

lim / e- {t - s) b(s)ds = 6(oo). (4.3) 

t->ooj 

Thus, Theorem |S] will be proved if one checks that 

limallT-^H = Vy e H. (4.4) 

To prove (|4.4|) one writes, using the spectral theorem: 

poo 2 

AT- l y\\ 2 = -—- 2 d(E s y,y) = \\P N yf = 0. (4.5) 
Jo \ s i a ) 

Theorem |3] is proved. □ 

Let us prove that the DSM method (j4.1j) yields a stable solution to problem (jl.lj) . 

Assume that / is replaced by fs, \\fg — f\\ < 5, in equation (j4.1|) and denote by ug{t) the 
corresponding solution. Then, using the estimate (|3.13|) . one gets 

lim \\u s (t 5 )- y|| = 0, (4.6) 

O—r0 

provided that 

\imtg = oo, lim — - = 0. (4.7) 

To check the sufficiency of the second condition ()4.7|) for ()4.6|) to hold, one proceeds as 
follows: 



\\us(t) - u(t)\\ = || I e-^T^A*(f s - f)ds\\ 

J 



(4- 



<6 I e -(ts)_^ ds< 6 



lo 2y/a(fi 

Here we have used the monotonicity of a(t), which implies a(t) < a(s) if t > s, and the 
estimate HT" 1 /!*!! < which was proved earlier. 
Let us state the result we have proved. 

Theorem 6. If tg is chosen so that ()4.7|) holds, then the solution ug(t) to problem (|4.1|) 
with noisy data fs in place of f satisfies (14. 6 j) . 
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5 A new discrepancy principle 



The usual discrepancy principle is described in Section ^ It requires solving nonlinear 
equation ||Aw 0j a — /|| = C5, C = const, 1 < C < 2, where u a> s = T~ 1 A*f$. Thus one has 
to know the exact minimizer u a ,s of the functional F(u) = \\Au — fs\\ 2 + gi||m|| 2 , or the 
exact solution of the equation T a u = A*f$. 

In this Section we discuss the following question: 

How does one formulate the discrepancy principle in the case when u a ,s is not the 
exact solution of the minimization problem F{u) = min ; but an approximate solution. 
Let us state the result. 

Theorem 7. Assume that A is a bounded linear operator in a Hilbert space H , that 
f = Ay, y _L N, \\fs — f\\ < 8, \\f&\\ > C8, C = const, C G (1,2), andu a ,s is any element 
which satisfies the inequality: 

F(u a , 5 )<m+(C 2 -l-b)6 2 , (5.1) 

where 

F{u) = \\Au- f 5 \\ 2 + a\\u\\ 2 , m=MF(u), (5.2) 

b = const > and C 2 > 1 + b. 
Then equation 

\\Au a , s -f s \\ = C6 (5.3) 
has a solution for any fixed 5 > 0, lim^ a (<5) = 0, and 

lim \\ug — y\\ =0 (5.4) 
where us = u a (s),s and a(S) solves (j5.r>j) . 

Proof of Theorem^ To prove the existence of a solution to ()5.3|) denote ||Au aj< 5 — fs\\ by 
h(S,a), check that h(S, +0) < CS, h(5,oa) > C5 and note that h(6,a) is a continuous 
function of a on the interval (0, oo). Then equation ()5.3|) has a solution a = a(S). As 
a — > oo one has 

a\\u a A\ 2 < F{ua )S ) < m + (c 2 - 1 - b)5 2 < F(0) + (C 2 - 1 - b)5 2 
so, with c := F(0) + (C 2 — 1 — b)5 2 , one obtains 



c 

\Ua,s\\ < — 7=, a —> oo. 

/a 



Thus 

h{5,oo) = \\A0-fs\\ = ||M| >C5. (5.5) 

As a — > 0, one has 

h 2 {5, a) < F{u a , s ) <m+{C 2 -l- b)5 2 < F{y) + {C 2 - 1 - 6)5 2 . 
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One has F(y) = 5 2 + a\\y\\ 2 , so h 2 (5, a) < (C 2 - b)5 2 + a\\y\\ 2 , and 

h(5,+0) < (C 2 -b)^5 < CS. (5.6) 

Finally, the continuity of h(5, a) with respect to a for any fixed 5 > follows from the 
continuity of the bounded operator a and the assumed continuity of u a ^ with respect to 
a. Thus, the existence of a solution a = a(5) > of equation (|5.3j) is proved. One takes 
a solution for which lim^o a(5). Such solution exists because h{5, +0) and m = m(S, a) 
tend to zero as 5 — > and a — > 0. 
Let us prove ()5.4|) . One has 

F(u s ) = \\Aus - fs\\ 2 + a(5)\\u s \\ 2 < \\Ay - f 5 \\ 2 + a(5)\\y\\ 2 < 5 2 + a(5)\\y\\ 2 - (5-7) 
Since \\Aus — fs\\ = C5, C > 1 it follows from (|5.7|) that 

ll^ll < Ibll- (5-8) 

Thus one may assume that us — ^ u as 8 — > 0. 

Let us prove that Au = f. First we observe that 

\\Aus - /|| < \\Au s - f s \\ + \\fs - f\\ < C6+ 5, 



so 



lim \\Au s - f\\ = (5.9) 
S— >o 



Secondly, for any v G H we have 



(/ - Au, v) = lim (Aita - Au, v) = lxm(u s - u, A*v) = 0, (5.10) 

5 — ^0 5 — >0 

because us — u. Since f is arbitrary, one concludes from f|5. 1(J|) that Au = f. From 
()5.8|) it follows that < \\y\\. Since the minimal-norm solution to equation Au = f is 
unique, one obtains u = y. Thus us — y, ||wa|| < ||y||- This implies (|5.4|) as follows from 

eng. 

Theorem [7| is proved. □ 



6 Discrepancy principle does not yield convergence 
uniformly with respect to the data 

In this Section we make the following assumption. 

Assumption A): A is a linear bounded operator in a Hilbert space H , N := N(A) = 
N(A*) := N* = {0}, A- 1 is unbounded, Ay = f, \\f s - /|| < 5, \\f s \\ > 5. 

Let a = a(S) be chosen by the discrepancy principle, 

\\Au a>5 - /a|| = CS, u 5 = u a{5 ),s = T^A*^. (6.1) 
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Consider the set Ss m . 

S 5 :={v: \\Av-fs\\ <5}. 

We are interested in the following question: given {fs,S}se(o,s ), where 5 > is a small 
number, and assuming that a(S) is the solution to ()6.1|) . can one guarantee uniformity 
with respect to the data f convergence? 
In other words, is it true that 

lim sup 1 1 — u|| = 0. (6.2) 

5 ~ >0 ves s 

The answer is no. 

Theorem 8. There exist fs such that 

lim sup \\us — v\\ > c > 0, c = const. (6.3) 

Proof of Theorem 8. Denote T~ X A* := G, u$ = Gfs, \\G\\ = We have proved in 

fll,10|) that \G\ < but the equality sign holds because in p.lOj) U is unitary under 
Assumptions A). Thus, one can find an element p = p a , \\p\\ = |, such that 

IIQ.II > illGII IH = g^. (6-4) 

Assumptions A) imply that the ranges R(A) and R(T) are dense in H. Thus one can 
find an element z = z a> s such that 

\\fs - AT b z - p\\ < -, be (0, 1), b = const. (6.5) 

For any v one has 

\\Gfs - v\\ < \\Gf s - GAv\\ + \\GAv - v\\. (6.6) 

Take v = T h z and let M > be an arbitrary large fixed a n constant. Then 

lim sup — u|| = 0, (6-7) 

S ~^° veS s ,v=B b z,\\z\\<m 



because 



\\GAv - v\\ = WT^Tv - v\\ = || - aT- l T h z\\ = a sup = ca\ (6.8) 

s >o s + a 

where c = b b (l — b) 1 ^ . 

From (J6.7)) and (J6.6|) one sees that 

lim sup \\Gfs ~ v \\ — 0- (6-9) 

<5 ^ D65 4 ,t) = T''2,||2||<M 
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if and only if 



lim sup \\Gf s -GAv\\ = 0. 

S ~^° v€S s ,v=T b z,\\z\\<M 



Take z — z a , v — T b z. Then 



\\f s -Av\\<\\ P \\ + \\f s -AB b z- P \\< 6 -+ 5 - 



55 



<6, 



and, using ()6.5|h one gets: 



IIGA-GA.II = ||G(/ i -A,-p) +Gp || > || GpH | G ||| > ± (I - 1) = ^ 



(6.10) 



(6.11) 



. (6.12) 



If ^ > c> 0, then, according to (l6~T2l . (I6~TU1) fails. 

Let us find /a such that for a = a(5), defined by the discrepancy principle, one has 
^ > c > 0. This will complete the proof of Theorem |HJ Let us assume for simplicity 
that A = A* > is compact. Then T = A* A = A 2 , and equation ()6.1|) becomes 



C 2 6 2 = ||L4(A 2 + a)-M-/]/, 



3=1 



*3 



L Aj + a 



- 1 



^ |2 -|J(A, + a) 2 - 



(6.13) 



Here Aj are the eigenvalues of A 2 , fgj = (fs,<fj), A 2 {pj = Xjipj, \\<fj\\ = 1- Assume, for 
example, that Xj = \ and \fsj\ 2 = -p. Then ()6.13|) becomes 



C 2 



y Ll 

U (r 1 + a) 



(6.14) 



Note that 
h : 



x~ 



(x- 1 + a) 2 
1 ds 



dx 



[s + a) 2 



[s + a 



,-i 



a- 1 [l-^] = -[l + 0(a)], a^O 
a + 2 a 



This and ()6.14|) imply -^>c>0as<5^0. Theorem |S] is proved. 



□ 



7 Iterative processes for solving equation ( 11 .11 ) 

In this Section convergent iterative processes for solving equation (jl.lj) are constructed 
in the case when A is a closed, densely defined in H, unbounded operator. Consider the 
process 

u n+1 = Bu n + T^A*f, ui = u lt Ul ±N, B:=aT-\ (7.1) 

where a = const > and the initial element U\ is arbitrary in the subspace N x , N : = 
N(A) = N(T), T = A* A, T a = T + al. Note that B > 0, ||S|| < 1. 
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Theorem 9. Under the above assumptions one has 

lim |K - y\\ = 0. (7.2) 

n— >oo 

Proof of Theorem^ Let w n = u n — y. Then 

w n+ i = Bw n = B n w, w := Mi — y, w _L N. (7.3) 

Let us prove that 



lim \\B n w\\ = 0. (7.4) 

n^oo 

If (17. 4|) is verified, then Theorem El is proved. We have 

H 5 "H| 2 = / 7 ™)= / +/ :=^i + ^, (7.5) 

Jo (a + sj J s>fc Jo< s <b 

where is the resolution of the identity corresponding to the operator T > 0, and 
b > is a small number which will be chosen later. For any fixed b > one has 
linv^oo J x = because < ^ < 1 if s > b. On the other hand, J 2 < f£ d(E s w,w), 

and lim b ^ J Q 6 d(E s w, w) — because if _L iV = -Eo-^- Therefore, given an arbitrary 
small number rj > one can choose 6 > such that J 2 < rj/2. Fix this 6 and choose n 
sufficiently large so that J\ < rj/2. Then ||5™w|| 2 < 77. Since 77 is arbitrarily small, we 
have proved (|7.4j) Theorem is proved. □ 

Remark 4. Iterative process ()7.1|) yields a stable solution of equation §1. l\i . Indeed, let 
fs be given, \\f$ — f\\ < 8, and let u n) s be defined by (|7. 1|) with f$ in place of f . Let 
w n ,s = u n j - y- Then 



w nuS = Bw n . s + T a A*(f s - f), so 



w n+hS = J2 BjT ~ lA *(fs -f) + Bn ^ u i - v), («i -v)±n. 

3=1 



(7.6) 



We have proved above that 

\\B n {u x - y)\\ := 8{n) -»■ as n -> 00. (7.7) 

One has: 

n 

J2 BjT « lA *(f*-f) 



^ (n + l)5 



3=0 

because \\B\\ < 1 and \\T~ 1 A*\\ < From f!7.7|) and ()7.8|) one finds the stopping rule, 
i.e., the number n(8) such that lim^ ||^n.(5),<y|| = 0. This n(8) is found for any fixed 
small 8 as the minimizer for the problem 

(n + 1)8 „. . . 

+ £(n) = min. (7.9) 



2^ 
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Alternatively, one can find rii(S) from the equation 



S („) = (7.10) 



Clearly n(S) and nx(8) tend to oo as 5 — > 0. 



8 Discrepancy principle for DSM 

In this Section we formulate and justify a discrepancy principle for DSM. 

Let us start with the version (j4.1j) . We assume that a(t) > is a monotonically 
decaying twice continuously differentiable function, lim t _ >00 [a(t) + |d| + a] = 0, a > 0, 
lim^oo = 0, for example, a(t) = , , where Ci, c and 6 are positive constants, 
b G (0.5, 1). For this a{t) all the assumptions (3.1) hold. 

Theorem 10. The equation 

\\AT-^A*f 5 - f s \\ =C5 C = const, 1< C < 2, (8.1) 

/ias a solution t = t$, lim^o^ = oo? stzc/i t/iat ()4.6j) holds, where u$(t) is the solution to 
(|4.1j) with fs in place of f , \\fg\\ > CS. 

Proof of Theorem\10\ We have proved earlier that equation (|8.1|) has a unique solution 
a = as and limg^o as = 0. If a(t) is a monotonically decaying function such that 
limt^oo a(t) = 0, then the equation as = a{t) defines uniquely t = ts, such that a(ts) = as, 
and lim^ot^ = oo. 

Let us sketch the proof of relation ()4.6|) . where us(ts) = f^ s e~( t5 ~ s ^T~,\A* fsds and 
ts — > oo as 5 -> 0. We have proved (cf (1.17)) that | IT^jA*/*!! < IMI, and 

lim I IT7. 1 , A* /<y -2/11 = 0. (8.2) 

(cf (1.18)). It is clear that lim^oo f* e~^~ s ^g(s)ds = g(oo) provided that g is a continuous 
function and there exists g(oo) := lim^oo g(t). 

Note that \im s ^ ts \\T; { l s) A*f 5 - T^A* f 5 \\ = 0. We have 

/ S e-^T^fsds = y + o(l) as 5 -> 0. (8.3) 

v 

To check this we use equation (8.2), the formula T~}s —T~^ = T~.^ [a(tg) — a(s)]T~, ts s, the 

estimates \\T~^A* f s \\ < \\y\\ and||T^|| < ^y, and the relation lim^^ f* e~^~ s) a(s ^ s " (t) ds = 
0, which holds due to our assumptions on a(t). Theorem 10 is proved. □ 
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